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1 Introduction 



We study phenomenological implications of the breaking of universality in weak 
interactions. Particular attention is paid to the signatures of new heavy ele- 
mentary vector particles {Z' bosons), which can be detected at the present or 
future high-energy e~^e~ and hadron colliders. As the quarks of the third family 
{t and b) are significantly heavier than those of the other two families, one may 
speculate that such a boson may be involved in the formation of this difference, 
and thereby interact directly with the t and b quarks (as well as the r and z/,-) 
and only much more weakly with the lighter quarks and leptons, solely due to 
radiatively-created mixings. 

Both the phenomenology of Z'-boson production and theoretical models for its 
dynamics are considered. A theoretical extension of the Standard Model adopt- 
ing a new heavy Z' boson may need to include more unknown heavy fermions 
and scalar Higgs bosons. Otherwise the quantum anomalies break the unitarity 
and the mass pattern for heavy fermions will violate experimental bounds on 
deviations from the Standard Model in the low energy data. 

There can be a variety of possible models depending on the strength of the 
vector and axial-vector couplings of the Z' to quarks and leptons. Additional 
bosons naturally appear in Grand Unification Theories jU E]. The left-right- 
symmetric model also contains at least one Z' boson jS]. 

Usually these Z' bosons interact universally with the fermions of all families; 
the phenomenology of such particles was considered by several authors P3 12111 El- 
There exists a scheme in which Grand Unification symmetry breaking leads to the 
formation of a so-called 'leptophobic' and 'hadrophilic' Z' boson, which interacts 
only with hadrons. The phenomenology of such bosons was also considered |2l IHl 
|7] . A related idea has been advanced by Okun and collaborators, who consider a 
'leptonic' photon jH]. 

A scheme with the breaking of universality in weak interactions has been stud- 
ied by Dyatlov (01 in connection with the problem of the fermion mass hierarchy. 
It was shown that a neutral gauge boson interacting differently with the heaviest 
fermions naturally leads to a realistic mass hierarchy and Kobayashi-Maskawa 
mixing matrix. The phenomenology of a Z' interacting only with the quarks of 
the third family was considered recently by Holdom ^Oj, by Frampton, Wise and 
Wright [m, and by MuUer and Nandi 

Irrespectively of the scheme of the Z'-boson embedding into an extended 
electroweak theory one should know the feasibility of detecting such a particle 
at a particular accelerator. Various high-energy colliders have been examined 
for that purpose: hadronic colliders (Tevatron and the LHC), LEP2 and future 
electron-positron colliders (NLC). 

For pp and pp colliders the main production mechanisms are the following: 
66-quark annihilation (from the small sea contribution in protons) near the Z' 
peak, quark-antiquark annihilation into a Z with conversion of the Z into a Z' 
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due to the small mixing and also gluon-gluon fusion. The Z' boson is assumed 
to be detected through an excess of hb or tt hadronic jets. Other processes are 
suppressed by the kinematics and due to small coupling constants. Numerical 
estimates have been made to determine whether the Z' signal can compete with 
the two-jet background originating from non-resonant processes. The conclusion 
is that the direct observation of a heavy Z' (heavier than the Z) with moderate 
coupling constants to h and t quarks is impossible at the Tevatron collider and 
unlikely even at the LHC. 

The most interesting possibility is to investigate the feasibility of a Z' discov- 
ery at LEP2 [13j and the next generation of e^e~ colliders whose design is now 
intensively discussed [T^. For these colliders the background of r+r" and two {bb 
or ti) jets is significantly lower than for the previous types of colliders; therefore 
there is a possibility of having kinematical windows for detecting a Z' boson. 

The paper is organized as follows: In Sect. 2 we set up the notation and dis- 
cuss constraints on the Z' from electroweak precision data. In Sect. 3 we consider 
production at hadronic colliders, and in Sect. 4 we consider e^e~ colliders. Sec- 
tion 5 contains a brief summary. In Appendix A we discuss the constraints on 
the (expanded) scalar sector due to anomaly cancellations, and in Appendix B 
we give some formulas on the QCD background. 

2 Model parameters and experimental input 

We consider a model with an additional massive neutral ?7(1) gauge boson Z' . 
The Lagrangian can be written in the form: 

where £sm is the full Lagrangian of the Standard Model (SM) and Cz' is an 
additional term: 

Here Cym is the usual free Yang-Mills Lagrangian for the Z' boson, ^Higgs the 
Lagrangian for scalar particles interacting with the Z' (one may consider different 
choices for the Higgs sector — some possible Higgs structures are discussed in 
Appendix A — see also [UllIS]), and £int specifies the interactions of the additional 
neutral boson with fermions. 

2.1 Additional NC interactions 

We assume that the Z' interacts only with fermions of the third family. Then 
the neutral-current Lagrangian is of the form j2]: 

-£nc = eJ^^A^ + gzJ^Z^ + gz'J'^Z'^, (2.1) 
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where the first two terms are just those of the SM and J'^ involves only the 
third- family fermions, t, 6, r and Vr- This form of £nc corresponds to the case 
where the new f/(l) neutral gauge particle is not mixed with the photon [llj. 

In the usual notation of the SM one has gz = g/ cos6'w, while gz' is a phe- 
nomenologically free parameter. However, in more complete theories involving 
Z' {i.e. GUT or LR models), the value of gz' is tightly connected to gz |21IZI: 



gz' = \l^svcLO^\f\gz, (2.2) 

where A is usually in the range 2/3 to 1 This means that the maximum value 
is 

gz'/gz ~ 0.62. (2.3) 

We shall use this constraint in most of the numerical work that is presented. 

Let us write down the Lagrangian for interactions of the Z and Z' with the 
fermions of the third family: 

-Czz'=gzZt, ipf-f^{vf-afj5)ijf + gz,Z'^ ^ ipfj'' {v'^ - a'f-f5) ipj. 

f = tbTUT f=tbTVT 

(2.4) 

The parameters Vf and aj are defined in the SM as follows: 

Vf = ^T^L - sin^ a/ = I^sl, (2.5) 

where T3 and Q are the third component of weak isospin and of electric charge, 
respectively (we suppress their flavour indices). The parameters v'jr and a'^ rep- 
resent the chiral properties of the Z' interactions with fermions and the relative 
strengths of these interactions. One of all these parameters for all fermions can 
be absorbed into gz', and in order to avoid this we normalise v'^"^ + a'^^ = |; this 
normalization was used in [Zj when obtaining Eq. ()2.2|1 . However, in our model 
the quantity f + a'^^ may vary from fermion to fermion. The cancellation of 
ABJ anomalies restricts the possible choice of v'j- and a'j- and will be discussed in 
Appendix A. 

The weak eigenstates and Z'^ may be mixed so that the mass eigenstates 

are: 



Zi \ _ f cos^ sin A / Z 
Zj = I -sine cosd [Z' 



(2.6) 



Then the well-known value of the SM Z-boson mass corresponds in this model to 
the Zi mass (Mi), while the mass of the Z2 boson (M2) is, of course, unknown. 
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It is possible to rewrite Eq. ()2.4|) in terms of the mass eigenstates: 



-^^ZiZz — 9z 



f=tbTi/T 

-Z2^, ^ ^/7^ - a/275) V"/ , (2.7) 

f=tbTU-r 



where 



Vfi = cos4 ffH sm^fr, afi = cost.af-\ sm^Or, 

9z ^ 9z ^ 

Vf2 = ^^cosCv'f — sinfvf, = cosf a'r — sinf a/-. (2.8) 

9z ^ 9z ^ 

The introduction of mixing between the Z and Z' changes the couphngs of the 
SM, which are extracted from the conventional experimental input. In fact, in 
the SM one has the standard electroweak input [THj of a(0), Gp [Ej together with 
Mz = (91.1863 ± 0.002) GeV ^H]- This input contains enough parameters for 
the determination of the SM electroweak coupling constants g and g' of SU{2)i^ 
and f/(l)y, respectively, and the Higgs vacuum expectation value v. In order to 
do this let us introduce the following quantities: 

^2 ^ ^ ^37 GeVf (2.9) 

V 2Cjf 



and [inHH! 



# (2.10) 



where Gp*" is the four-fermion NC Fermi coupling constant in the limit of zero 
momentum transfer. At the tree level the p parameter can be defined as po = 
M^/(M| cos^ 6*-^) and in the SM po = 1 due to the doublet Higgs structure, 
while at the one-loop level it acquires radiative corrections |19j^ 

1 3G 

p^- ApT^-^m^^O.Ol. (2.11) 

1 - ApT 87r2V2 

Then it is possible to express the couplings of Eq. ()2.4|1 in terms of the above- 
mentioned input 1^: 

gz = {aV2GfMIpY/\ (2.12) 



-'^For hh final states there are extra vertex corrections 
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i_JfL^. (2.13) 

It should be stressed that the angle introduced above differs from the conven- 
tionally defined Weinberg angle: 

The one-loop expression for this difference can be found in fTI\ . 

In the extended model the mass of the weak eigenstate Mz [i.e. the matrix 
element of the mass operator) is unknown: it is related to the mass values Mi 
and M2, and to the mixing ^, as 

As a result, for non-zero mixing angles there are corrections to the Weinberg 
angle o^nd gauge couplings g and g' . They acquire dependence, now not only 
from the standard electroweak input, but from the parameters of the extended 
model M2) as well. This means that the results of a measurement of other 
electroweak quantities may be used to restrict these parameters. 

Following one finds that in the extended model Eq. ()2.1H|1 should be 
modified by Mz — > Mi, while the p parameter acquires additional contributions 
at the tree level: 

• from the ZZ' mixing (Apm): 

, /Ml \ 1 + (Ml/M^) tanV 

• from the additional Higgs sector — related to the pattern of symmetry break- 
ing (Apse) (the scalar sector of this theory necessarily contains additional 
Higgs fields), 

so that: 

1 

p = PSB ■ PM- PT= i_ 

Ap = ApsB + ApM + Apr, (2.17) 

where psB and pu contain tree-level corrections due to the extension of the model, 
whereas pt contains the SM radiative corrections (dominated by the top quark). 
Of course, in the extended Z' model there exist additional radiative corrections 
to the p parameter, which can even be rather large since there is no decoupling of 
the heavy particles for these corrections, but we restrict ourselves by considering 
only the tree-level effects of the non-standard particles. 

If the scalar sector of the theory contains only doublets and singlets of the 
SU{2)i group one has ApsB = 0. In further considerations we shall use this 
assumption by default. 
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2.2 Existing bounds on the Z' mass 

A Z' would contribute with full strength to the decay of T states to r^r^, 
suppressed only by the Z' propagator. The moderate agreement of the branching 
ratio [T2j 

with 

BR(e) = 0.0252 ± 0.0017, BR(/i) = 0.0248 ± 0.0007, (2.19) 

implies that the Z' mass cannot be too low. 

Let us estimate this lower bound. Since the T(IS') is a natural-parity state, 
only the vector part of the hadronic current contributes to its decay. Hence, no 
restriction can be obtained for the pure axial Z'\k vertex without ZZ' mixing 
(such mixing effects are small, and will be omitted from the discussion of T(15') 
decay). For couplings involving the vector current, one may obtain a lower bound 
on the Z' mass: Mz' > 50 GeV for vector-vector chiralities (in the Z'bb and 
Z't^t^ vertices), Mz' > 22 GeV for the vector-axial- vector case, and Mz' > 
35 GeV for the LL, RR, LR and RL cases. 

Another kind of restriction can be obtained from the p parameter, as discussed 
above. If one combines the LEP results with the My//Mz value, it is possible to 
conclude that 

2 • 10"^ < Ap < 8 ■ 10"^ for 150 GeV < rrit < 200 GeV (2.20) 

and 70 GeV < rriH < 1000 GeV. Under the assumption that ApsB = 0, this 
leads to 

1 < PM < 1.005. (2.21) 

The lower bound on pM implies that M2 > Mi. 

However, one should keep in mind that these constraints are valid only under 
the assumption that psB = 1? which requires a certain simple structure of the 
Higgs sector. For a more general scalar sector one has 



^'"^ " 2V (hUt^IH) ■ ^ ^ ^ 

For Higgs fields with isospin higher than 1/2, psB may be less than unity and the 
above-mentioned constraints are removed. 
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3 Hadronic production 



The hypothetical Z' boson could be produced in proton-proton (antiproton) col- 
lisions through direct Drell-Yan-like coupling of 6 and h sea quarks, through 
gluon-gluon fusion, with the gluons coupled to 6 or t quark loops, as well as 
through ZZ fusion and WW fusion. We shall consider these mechanisms in turn 
in the following, focusing first on the prospects for producing such vector bosons 
at the Fermilab Tevatron, and then turn to the LHC in Sect. 3.4. In this section 
we consider the limit of no ZZ' mixing, which implies Mz' = M2. Also, we shall 
here disregard the constraint ()2.3p . letting Z' couplings be of 0{1). 



3.1 Direct production by bb annihilation 

The vector boson Z' may be produced by the direct Drell-Yan mechanism, i.e. 
annihilation of a 6 quark from one proton (or antiproton) and a b from the other. 
The matrix element is proportional to 

(3.1) 

where pi and p2 are the relevant parton momenta of the incident proton and 
antiproton, with k = Pi + P2, and q and q the final-state quark and antiquark 
momenta. The corresponding polarisation-averaged square becomes 



|M|^ 



^9% 



(F - M|,)2 + [Mz'Tz' 



:{Xo + mlXb + m^Xq + mlmlXbg 
+ mtXbb + mlXqq}, (3.2) 



where m, 



q l- 

Xo 
Xb 

Xbq 
Xbb 



rUb or rrit) denotes the mass of the final-state quarks and 

= + 0« + + + Ha'bV',a',s{i - u), 

= {<' + <'Mv'b' + a'b'){s + 2i) + 5] + 12t;^aVa;S, 



{v',' + a'b'mv'q' + a'/)(s + 2t) + 2v'q' s] + I2v'ba'bv'qa'q s 
^{v'b'v'q' + a'b\'[^ - 2{s/Ml,) + {s/Ml,m, 
X,, = 2{v',' + a',W,' + a'q% 



(3.3) 



with -s = k'^, i = {pi — g)^, and u = {pi — qY . 

For the decay of the Z' (of mass Mz') to a fermion-antifermion pair (of mass 
mj), we find the partial decay width at the tree level 



g%Mz 

\2tx 



(2mf/Mz' 



.1 2 



1 + 



2m) 
Ml 



I ' 2 



Am) 
Ml 



(3.4) 
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where v'^ and a'j denote the vector and axial couphngs, respectively. Thus, the 
total fermionic width would be given by 

Tz' = + + + r,^ . (3.5) 

The inclusive cross section may be expressed in terms of a convolution integral 
over 6-quark distribution functions: 

Ux, ! dxo ^f^, fi{xi)fl{x2). (3.6) 



di?j_ J J d-E_Ldxidx2 

with E±_ the transverse energy of the jets, and f1{x) the h (6)-quark distribution 
functions. The elementary cross section is given by 



with |M|2 the square of the matrix element in Eq. ()3.2j) . properly averaged and 
summed over polarisations, and 4i?ii?2Vrei = 2{pi + ^2)^ = 2s. Straightforward 
kinematical considerations lead to: 



d^o- 1 1 — — r E 



|M|2 = ± , (3.^ 



dE±dxidx2 27r 8P/^ Je^~ El coshy - E.sinhy 



where y is the rapidity and 
E, 



Er, 



1 = 175 coshy ± sinhy^l - AEl/s . (3.9) 



It is convenient to replace the integration over xi and X2 by one over rapidity 
and s, the invariant mass squared of the Z' . With r = s/s, where s is the 
squared cm. energy of the proton-antiproton system, we have Xi = y/re^ and 
X2 = V^^~^) obtain 

dy\M\^f1ix,)f,ix2) 



dE^ 27r 8s 7 s 7 y/^^^ cosh 7/ - sinh |/ ' 

(3.10) 



where |M|2 is obtained from Eq. ()3.2|) . 

Resulting cross sections are given in Fig. 1 for the Fermilab energy, ^/s = 
1.8 TeV, for three values of the mass, Mz' = 100, 200 and 400 GeV. We here 
disregard the constraint ()2.3|) . taking the more 'unbiased' view that gz'O-' and 
gz'v' are of 0{1). (If we adopt Eq. ()2.3p . the cross section would drop by about 
two orders of magnitude.) For the distribution of b (6) quarks in the incident 
protons and antiprotons, we use standard values |2^ l25] . For comparison, we 
also show the dominant QCD contributions [cf. Appendix B] and data (summed 
over all flavours) [26j. The direct bb production through a Z' is seen to be below 
the QCD rate by 3-4 orders of magnitude, even for a 'light' Z'. 
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3.2 Gluon fusion 



In the collisions of protons and antiprotons at Fermilab (or protons at LHC), 
gluon fusion may, via a suitable quark triangle diagram, lead to production of 
such Z' bosons. The inclusive cross section may be expressed in terms of a 
convolution integral over gluon distribution functions, similar to Eq. ()3.6|) : 

with /f (x) gluon distribution functions. The elementary cross section is given by 
an expression similar to Eq. ()3.7j) . where the amphtude for the production of qq 
{bb or tt) final states may be expressed as 

M = ^A6"(l)e^(2) gz> _ jj^ + zmJ,, ' <^M<l)] ■ (3.12) 

The factor V^l^^ describes the Z'gg vertex (a and b are colour indices), which we 
take from the corresponding expression for the Zgg vertex, given by Kniehl and 
Kiihn P?]. 

By current conservation, k'^[u{q)'yiyv{q)] = 0. Furthermore, for on-shell gluons, 
the vertex function simplifies considerably. Averaging over gluon polarisations, 
and summing over q and q polarisations, we obtain 

Here, a'g describes the axial coupling of Z' to the quark field of the triangle 
diagram, normalised as in Sect. 2. (If we were to replace the Z' by the ordinary 
Z, we would get a'qgz' — > 6X3/(2 sin cos ^w); with T3 = ±| the quark isospin.) 
The coefficient Bq is given in Ref. [2Z1- For a triangle diagram of quark flavour 
Q, it is 

Bq = ^{1- 2mlCo) , (3.14) 

with A = A(si, S2, s) = the Kallen function (for on-shell gluons, si = S2 = 0), 
niQ the mass of the loop quark, and Cq given as 

Co = ^{2cf>-nf, sin0= ^Aml/s-l, s < 4ml, (3-15) 
Co = (^og -iTT^ , a= ^l-Am'^/s < 1, 4mJ < s, 

depending on whether s is below or above the threshold associated with the loop 
quark. We consider the contributions from both b and t quarks to this loop: 

a'Q'\BQ\'' ^\a[B, + a[Bt\\ (3.16) 



10 



The result fj3.13|) is proportional to a'^ni?^. According to Furry's theorem, the 
vector part of the triangle diagram cancels, and the remaining axial anomaly is 
proportional to the final-state quark mass, here denoted by m^. At moderate 
energies, the relevant final states are hb, thus ruq = rrib. If the energy is high 
enough, there is a similar contribution for tt final states. 

Replacing the integrations over xi and X2 by integrations over rapidity and 
s, the invariant mass squared of the Z', one obtains an expression similar to 
Eq. ()3.10|) . where the b and b distribution functions should be replaced by gluon 
distribution functions, and where |Mp also should contain a factor 1/8 from 
colour matching of the two gluons. 

Numerical values are obtained for these cross sections, using standard gluon 
distribution functions [23 • The resulting cross sections are for the Tevatron 
energy shown in Fig. 2. We have arbitrarily taken the axial couplings to b and 
t quarks to be the same. The cross section is remarkably small, even for moder- 
ately low masses, Mz' — 0{Mz)- The resonant structure is due to interference 
between the contributions of b- and t-quark triangle diagrams, cf. Eq. ()3.16p . This 
interference is illustrated in Fig. 3, where for Mz' = 200 GeV we compare three 
cases: (1) the bbZ' and ttZ' axial couplings being the same (solid), (2) opposite 
(dashed), and (3) the ttZ' axial coupling being zero. It appears that no variation 
of the chirality of these couplings can make the cross section comparable with 
the QCD background. 

3.3 Fusion of ZZ or WW bosons 

The fusion of two weak gauge bosons, like that of two gluons, can also lead 
to Z' production. This could proceed through mixing, or via a triangle loop. A 
crude estimate for the magnitude of this rate may be obtained in the Weizsacker- 
Williams approximation. The resulting cross section is found to be several orders 
of magnitude below the Drell-Yan rate. 

3.4 Production at the LHC 

At the LHC the sea-quark and gluon distributions are much less suppressed than 
at the 'lower' energy of the Tevatron, so it is of considerable interest to see if the Z' 
production can compete with the QCD background. We show in Figs. 4 and 5 the 
cross sections for bb Drell-Yan-type production and gluon fusion, respectively. As 
at lower energies, it is the Drell-Yan-type production that dominates the gluon- 
fusion mechanism and, relative to the QCD rate, the Z' production is now 'only' 
suppressed by about two orders of magnitude. One must conclude that it would 
be extremely hard to discover such vector bosons in hadronic collisions. 
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4 Electron— positron annihilation 



In e~^e~ collisions the Z' boson could be produced directly via mixing with the Z 
boson; the following channels with two-particle final states are sensitive to this 
mixing: 

e+e" bb; e+e" tt; e+e" rr. (4.1) 

The first of these was discussed in |71 E|- In the following subsection we are 
going to discuss all these processes and calculate the relative deviation of the 
cross sections from their values in the SM for different values of the mixing angle 
^ and the Z2 mass M2. 

For the processes with more than two particles in the final state we will 
consider the process e'^e~ bbVf.Uf, as one of the most important for LEP2 and 
future colliders. 



4.1 Two-fermion final states 

We first consider the processes e^e" bb, ti, t~^t~ as the simplest processes 
at electron-positron colliders involving fermions of the third family, where the 
effect of Z' exchange could be observable. Three Feynman diagrams describe 
this process, with the exchange of photons, Zi and Z2. Deviations from the SM 
will occur only in the case where the mixing angle is non-zero. (Without tree- 
level mixing, it will arise only through loop effects.) The effect of ZZ' mixing 
changes the couplings of the Z boson and gives rise to Z2 exchange. The coupling 
between the electron and the Z' is proportional to sin,^, which by assumption is 
small. On the other hand, this mechanism has the advantage that there is no 
suppression by the Z' propagator and the effect might thus be observable. 
The cross section is given by the expression (for m/ <^ y/s): 

a,/=^F„ (4.2) 

where 

^1 = QlQ} + 2QeVeiQ fVf iRexi + (vj^ + ali){v% + a}^)\xi\^ 

+2QeVe2Q fVf 2ReX2 + iVe2 + ae2)(^^/2 + 42)1^2^ 

+2{VelVe2 + aeiae2) {v fiV f2 + a/ia/2)ReXiX2* > (4-3) 

with 
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and [cf. Eq. d^ ] 



Vi = — Vi, ai = — tti. (4.5) 

e e 

The ratio gz/e should be extracted from the standard electroweak input discussed 
in Sect. 2, and include the 'running' dependence from the e'^e~ energy ^/s. 
For y/s = Ml one can use the following [13 CEl 



^(Mi^) 
e 



1 






sin 6'w cos 







1/2 



(4.6) 



For ^/s > Ml we will start with ()4.(j|l and then use the solutions of the one- 
loop massless renormalization-group equations for the SU{2) and U{1) running 
couplings, g and g': 



167r2 3 Mf) ' 
9'^(s) - AMf)(l-<^?>og^V. (4.7) 



167r2 3 ° Ml 



which are related to e and sin as 



^ = 2 I /2 ' = 2 I /2 - 

It should be emphasised that all the above-mentioned couplings depend, for the 
model under consideration, on the mixing angle ^ and the mass M2 through the 
p parameter, cf. Eqs. ()4.6|1 and ()4.7|1 . 

No deviation from the SM has been observed at LEP. One may thus obtain 
bounds on the model parameters ^ and M2, taking into account available data on 
the Z° peak. For LEP2 and the NLC we take a rather conservatively anticipated 
precision. The sensitivity of observables, e.g. of the total cross section a^f, has 
been assessed numerically by defining a function as follows: 

where ^cr^j = a — a'^^ and the uncertainty is the statistical one. As a 
criterion to derive allowed regions for the coupling constants if no deviations from 
the SM were observed, and in this way to assess the sensitivity to the parameters 
^ and M2, we impose that < Xcrit; where Xcrit is a number that specifies the 
desired level of significance. 

At the peak, the most sensitive quantity is the forward-backward asym- 
metry. The resulting allowed bounds on ^ and M2, at the 95% C.L., are given 



13 



for bb and r+r^ production in Figs. 6-8, for different assumed chiralities of the 
coupling to the Z' (vector, axial, left, right). The relative gauge coupling is cho- 
sen according to j2] as Qz'/qz — 0.62. These bounds were obtained from the data 
reported in ^01, by means of the program ZEFIT, which has to be used along 
with ZFITTER 

In these same figures, we also present bounds corresponding to conservatively 
assumed cross-section precisions of 5% and 10% at LEP2 and a linear collider 
operating at 500 GeV (labelled NLC). These cross sections were calculated by 
means of the CompHEP |3ZI program. The qualitative difference between the 
LEPl and the LEP2 (or NLC) contours is due to the following. At LEPl (on 
resonance) there are three effects: modification of the couplings due to mixing, 
modification of sin^ ^w, and a shift of p from the SM value. The correspond- 
ing bounds are smooth curves. At higher energies (LEP2 and NLC) there is an 
additional contribution mediated by Z2 exchange. At relatively low M2 values, 
the Z2-exchange contribution dominates the deviation from the SM, whereas at 
higher M2 this contribution becomes less important. The complicated shape of 
the contours is due to interference between direct {Z2 exchange) and indirect 
effects. These effects interfere constructively at some values of sin^, and destruc- 
tively at others. 

In our input scheme, for a fixed non-zero value of ^, the gauge coupling gz in- 
creases with M2. This is due to the increase of pm, and hence of p [cf. Eqs. fl2.12|l . 
(I2.16|) and ()2.17|) ] with M2. This leads to a deviation of the cross section away 
from the SM value. Thus, at large values of M2, the e^e~ ff cross section 
is seen to impose strong constraints on the allowed mixing angle; there is a nar- 
rowing, at large M2, of the allowed region in sin^. (If we had frozen p and gz 
in our calculations, then the different points in these figures would correspond to 
different choices of Higgs sector — which would be rather unnatural.) 

For the case of bb final states, we consider two different centre-of-mass ener- 
gies, y/s = 190 and 500 GeV, in Figs. 6 and 7, respectively. The sensitivity of 
LEP2 and the NLC to ^ and M2 depends on the chiral property of the Z' cou- 
pling. For vector-, axial vector- and left-handed couplings, LEP2 (and the NLC) 
will have more sensitivity than LEPl at negative values of ^. For right-handed 
couplings, the situation is reversed. Thus, LEP2 and the NLC have the potential 
to observe effects of such a Z' in this channel with masses up to the order of 
1 TeV. Concerning r+r" final states, it appears that studies at higher energies 
cannot improve on the results obtained at LEPl, see Fig. 8. 

In Fig. 9 we consider tt final states at ^/s = 500 GeV. As compared with bb 
final states, the sensitivity is lower. However, the interference effects are different, 
and it is therefore complementary to the bb channel. 

Comparing the contour levels for the different final states, one can see that 
the tightest restrictions are obtained from the bb final state, while the t~^t~ 
case is the least restrictive. This difference can be understood from an analysis 
of the expression ()4.3|) . In comparing with the SM cross section, the important 
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difference between the final-state fermions is the electric charge, which dominates 
the main SM contribution. Thus, for a given value of (^, M2) away from the 
resonance ^/s = M2, the relative deviation from the SM will be largest for b 
quarks (because of the small electric charge) and smallest for r leptons (because 
of the large electric charge). 

4.2 Four-fermion final states 

We now turn our attention to the process e^e~ —>■ hbVf.Vf., which is sensitive to the 
three-boson couplings. In the SM, there are in the unitary gauge 23 Feynman 
diagrams that contribute to this process. In the extended model, the number 
of diagrams is 48 (in the unitary gauge), even if all the scalars are excluded. 
However, deviations from the standard model occur only for the case of a non- 
zero ZZ' mixing As a result of this mixing there appear three-boson couplings 
involving the Z2. 

Experimentally this process can be investigated by the detection of the hb pair 
with missing energy. In this case one measures the cross section of the processes 
e^e~ —>■ bbuiUi, integrated over a suitable kinematical region (with the neutrinos 
of all three families in the final state). However, following the work of Ref. |33j . 
we believe that the main contribution comes from the process with the UgUe pair 
in the final state. 

The process has been studied using CompHEP, which generates the Feynman 
diagrams (we omit virtual Higgs particles) and evaluates the cross section. This 
has been integrated over phase space according to the cuts of Ref. jHS] • For the b 
jets to be detectable, we require them to have sufficient energy, to be away from 
the beam pipe, not too close to each other, and not have an invariant mass close 
to the Zi. Furthermore, the missing momentum should have a large transverse 
component and a low rapidity, and the undetected neutrinos should not originate 
from a Zi. 



At a given energy, the cross section increases significantly with increasing 
values of M2, and also with | sin^|, as is shown in Fig. 10. The increase in the 
cross section seen at increasing values of M2 is basically due to the fact that the 
coupling gz increases through the increase of the pu parameter, as was discussed 
in Sect. 4.1. 

^There is no tree-level WWZ' coupling since we assume the Z' is an SU{2) singlet. One 
should keep in mind that the one-loop Feynman diagrams give rise to a WWZ' vertex even if 
at the tree level the mixing angle vanishes. 



Eb > 20 GeV, 
Ob-b > 20°, 
> 40 GeV, 



20° <eb< 160°, 
1^65 - m^J < 3Tz 



(4.10) 
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For the cases of y/s = 190 and 500 GeV, and for vector couplings, the modi- 
fications of the cross sections, with respect to the Standard Model, are given in 
Figs. 11 and 12, respectively. The gross features of these figures are rather similar 
to those for the bb final states, and the sensitivity is quite comparable. 

5 Concluding remarks 

We have shown that a Z' boson coupled only to the third-family fermions is 
rather difficult to discover, even at high-energy colliders, thus confirming the 
more exploratory analysis of Ref . ^1] . 

For hadronic colliders, such as the Tevatron or the LHC, this Z' is invisible 
because of the QCD background, which is many orders of magnitude greater than 
the cross sections involving the Z'. The data available from LEPl already exclude 
significant regions of the parameter space (^, M2). LEP2 and future e^e~ linear 
colliders can improve on these bounds, in particular from studies of final states 
involving bb. 

It seems that some additional progress may be achieved in the study of pro- 
cesses with four fermions in the final state, if one investigates not only the full 
cross sections, but also their dependence from the bb {ti, rf) invariant mass. We 
hope to return to this question in future work. 
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Appendix A: Higgs sector and anomalies 



Since we consider our model as an extension of the Standard Model, it should 
contain the standard Higgs doublet: 



^2 + ^ 

where v is the vacuum expectation value. This field realizes the representation 
(1/2,1,0) of the gauge group SU{2)i, x C/(l)y x C/(l)y/ and hence has the following 
covariant derivative: 

D^H = (^d^ + igT'^W; + i^-B^ H. (A.2) 

The kinetic term for this field contributes to the mass terms of the and 
Z bosons: 



This field does not supply the additional gauge boson Z' with a mass. It is 
impossible to assign to the field H a non-zero Y' hypercharge: in this case the 
mass terms for the fermions of the first two families will lose the U{1)y' invariance. 
Therefore = and in our model the field H could give rise to the masses of the 
third- family fermions only in the case when Yj:-^ = Yj^^ (not necessarily zero). In 
the general case, when Y^'l ^ Yj-^, the ordinary Higgs field H does not contribute 
to the masses of the third- family fermions. 

The simplest way to obtain a Z' mass is to introduce a scalar singlet 0, which 
transforms as (0, 0, Yg) and has a covariant derivative: 

D,4>= {d, + igz^Z'^4>. (A.4) 

With a non- vanishing vacuum expectation value, (0)vac = "^s/^/^-, it produces the 
following mass term for the Z' boson: 

\D,ct>\' =^ 'fy's'vlZ'^'- (A.5) 

By means of this field 0, an arbitrary mass can produced for the Z' boson, 
but it is not possible to get ZZ' mixing at the tree level. In the case where the 
Z' boson is not a pure vector particle \Yj^ ^ F^'j^j , it is also impossible to obtain 
gauge-invariant mass terms for the fermions of the third family (see above). 
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In order to solve these two problems, let us introduce an additional Higgs 
doublet Hi, which interacts with the Z' and hence has a non-zero Y' hypercharge. 
This field transforms as (|, 1, Y() and has a covariant derivative: 

D,Hi = (^d, + zgT^W^ + i^-B, + i^-fY[Z')j H^. (A.6) 

(The value of Y/ is not arbitrary when the field Hi gives rise to fermion masses, 
this will be discussed at the end of this appendix.) With (-ffi)vac = vi/\/2, it 
produces the following contribution to the mass term of the neutral gauge bosons: 

\D,Hi\^ =^ -4^"j (zl + % sin^ e^Y;%^ - 2 ^ sin e^Y;z,zA . 

COS"' 8 \ ^ g'^ ^ g' / 

(A.7) 

So in this case when all three Higgs fields are present, one obtains a mass matrix 
for the neutral gauge bosons in the general form: 

where 



Ml = J^^iv^ + Ml, = '^{vlYl + vlY^l (A.9) 

Mlz' = l^x^sin^wn. (A.IO) 

cos^t^w 4 g' 

After the diagonalization of this matrix, the mass eigenstates Mi, M2 and the 
mixing angle ^ can be expressed through v.e.v.'s of the introduced scalars. The 
field Hi also gives masses to the third-family fermions. 

However it should be stressed that the values of the Y' hypercharge are not ar- 
bitrary: restrictions come from the triangle anomalies. The well-known condition 
of ABJ anomalies reads 



Tr(T,{T,-,rfc})L = Tr(T,{r„Tfc})R (A.ll) 

where T are the matrices of the fermion representations, and i, j, k may refer to 
different subgroups of the full gauge group. 

In the Standard Model with the group 5'f/(3)c x SU{2)i^ x [/(l)y, these con- 
ditions unambiguously fix ratios of electric charges (or hypercharges) of fermions: 
four independent conditions of diagrams with LLF, YYY , ccY and ggY external 
fields (here L denotes the SU{2)i^ field, Y the f/(l)y field, c the SU{3) field, 
and g the graviton) for four ratios of Y/l, ^eR, ^gL, ^«r, have a single so- 
lution. For the extended model, there are additional conditions: LLy', YYY\ 
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YY'Y\ Y'Y'Y', ccY' and ggY' , which necessarily lead to Y' = Y. All other Y' 
assignments need an extension of the fermionic sector. In order not to introduce 
exotic fermions let us consider the simplest extension of the fermionic sector, 
namely addition of right-handed neutrinos. In this case the anomaly cancellation 
conditions, involving the Z' boson read: 

LLF' : + Sr^L = 0, 

ccY' : 3(2r^L - >^/r " ^r) = 0, 

ggY' : 2Y^^ - Y;^ - Yl^ + 3(21^4^ " ^^'r - ^r) = 0, 

Y'Y'Y' : 2Yi^ - Y;^' - + 3(21^4^=^ - - ^r^) = 0, 

YYY' : 2F^L - ^Y^k + 3 (^gl^^L " y^R - ^nn) = 0, 
YY'Y' : -2F^l' + 21;'r' + 3 0^)^' - ^F/r^ + ^F.'r^^ = 0. 

(A.12) 

Here, C and Q denote the third-family lepton and quark doublets. The last two 
equations are not independent. With Xi = Y- /Y^^, the system has the following 
solution: 

_ 1 

XCL — 1, a^QL — — XrR — 2 — XyR, 

2 4 

XbR = -- Xi,B., XtR = -3 + ^!^R) (A.13) 

where FJ^ (or x^r) remains undetermined. For the case x^r = one gets Y = Y' 
(case of the SM), whereas x^r = 1 corresponds to a purely vector Z' boson; the 
general case can be treated as a linear combination of these two cases. 

These values of fermionic hypercharges determine the hypercharge Y( of 
the Hi field: the gauge invariance of the fermionic mass term necessarily leads 

to Yi = y;^ - y;^. 

A vector-like Z'-fermion coupling is achieved with a universal choice: Xi^r = 1. 
However, as follows from Eqs. ()A.13|) . the assignment for the Z'-fermion vertex 
to be axial, left- or right-handed cannot be done universally. For instance, the 
coupling may be purely axial either for b quarks, or for r leptons, or for t quarks, 
but not simultaneously. 

Let us now comment on the chirality choices considered in Figs. 6-9. For 
the Z'bb coupling (Figs. 6 and 7) to be vector, axial, left- or right-handed, one 
must choose x^r = 1, 1/3, 2/3, or —00, respectively. For the Z't^t~ coupling 
(Fig. 8) to be vector, axial, left- or right-handed, one must choose x^r = 1, 3, 2, 
or —00, respectively. For the Z'ti coupling (Fig. 9) to be vector, axial, left- or 
right-handed, one must choose x^r = 1, 5/3, 4/3, or 00, respectively. 
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It may be useful to notice that the cancellation of anomahes is necessary only 
if the additional Z' boson is treated as a fundamental particle. For a composite 
Z\ arbitrary values of fermionic hypercharge are allowed. 



Appendix B: QCD background 

We shall assume that light-flavour quark jets can be rejected and do not constitute 
a background. Thus, the background of interest is the one due to U) (and, at 
sufficiently high energies, ti) jets. 

B.l Quark— antiquark annihilation 

With Pi + p2 = ^' + ^, s = (pi + ^2)^, the matrix element can be written as 

M = v{p^)^^gT,u{p^)'^u{h)^''gT^v{h) Sab, (B.l) 

s 

with g the QCD coupling. Properly averaged over spin and colour, the squared 
matrix element takes the form: 

JW=l ^{2{i - m\ - mlf + 2{u - m\ - mlf + {m\ + m^)5}, (B.2) 
where i — — b)^, u — {pi — b)^, and Pi — P2 — "ml- 

B.2 Gluon fusion 

Gluon fusion will also lead to hb (or tt) jets. There are three diagrams contribut- 
ing, including the one due to the three-gluon vertex [pi + p2 — p — b + b): 

M = + + M3, 
M, = u{b)igi,Ta f!~^\:^'^'\ tgi2nv{b), 



{b — piY — m 



M2 = mwm '^l ^i^'^'l igiMb), 

\b-p2) - ml 

Ms = u{b)ig^^%v{b) r— 

p^ + le 

x{-'i9)r^''[9al3{Pl - P2)u + gpv{p2 + p)a " QuaiV + Pl)/3]ei 63 • 

(B.3) 

Since the final-state quark and antiquark are taken to be on-shell, the gauge- 
dependent part of M3 vanishes. Properly averaged over spin and colour, the 
squared matrix element takes the form 

|M|2 = ^^[X + F + Z], (B.4) 
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where X is due to Mi and M2, Y is due to their interference with M3, and Z 
represents M3 squared. Furthermore, 

(i-m^)2 {t - ml){u - mf) (u - mlf 



with 



Similarly, 



with 



and finally 



Xtt = -[{i + ml){u-ml) + 2ml{s + 2ml)\, 
2 

= --ml{s-Aml), 
Xuu = \[{i-ml){u + ml) + 2ml{s + 2ml)\. (B.6) 



F = Fi + F„, (B.7) 



1; = Yt{i^u) (B.8) 
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Figure 1. Cross sections for inclusive jet production at the Tevatron, pp colli- 
sions at Ecm = 1.8 TeV. The solid curves represent Drell-Yan-type production 
of Z', from b and b (sea) quarks in the initial state. Three masses are considered, 
Mz' = 100, 200 and 400 GeV. The couplings are: gz'v'^, = gz'a'^ = 1. Also the 
contributions from the dominant QCD mechanisms are shown, as well as data 
(summed over all flavours) 
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Figure 2. Cross sections for inclusive jet production at the Tevatron, pp col- 
lisions at = 1.8 TeV. The solid curves represent production of Z', from 
gluon fusion. Three masses are considered, Mz' = 100, 200 and 400 GeV. 
Both b and t quarks contribute to the triangle diagram, with equal couplings, 
gz'd't — Oz'd'b = 1- Also the contributions from the dominant QCD mechanisms 
are shown, as well as data (summed over all flavours) . 
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Figure 3. Similar to Fig. 1, but for Mz' — 200 GeV only. Three cases of 
qqZ' couplings are considered: {gz'd't-i gz'd'i^ — (1, 1) (solid), (0, 1) (dotted), and 
(1,-1) (dashed). 
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Figure 4. Cross sections for inclusive jet production at the LHC, pp collisions 
at Ecm = 14 TeV. The solid curves represent Drell-Yan-type production of Z', 
from b and b (sea) quarks in the initial state. Three masses are considered, 
Mz' = 100, 200 and 400 GeV. Also the contributions from the dominant QCD 
mechanisms are shown, using standard distribution functions 
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Figure 5. Cross sections for inclusive jet production at the LHC, pp collisions at 
Ecra — 14 TeV. The solid curves represent production of Z', from gluon fusion. 
Three masses are considered, Mz' = 100, 200 and 400 GeV. Both b and t quarks 
contribute to the triangle diagram, with equal couplings, gz'dt — Oz'^^b — ^■ 
Also the contributions from the dominant QCD mechanisms are shown. 
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Figure 6. Allowed regions of sin^ and M2 obtained from LEPl data (95% C.L.) 
for the process e+e~ — > hh. Also shown are bounds anticipated from LEP2 at 
levels of assumed precision as indicated by labels. Four different chirahties are 
considered. 
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Figure 7. Allowed regions of sin,^ and M2 obtained from LEPl data (95% CL.) 
for the process e+e~ — > 66. Also shown are bounds anticipated from 500 GeV 
at levels of assumed precision as indicated by labels. 
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Figure 8. Allowed regions of sin^ and M2 obtained from LEPl data (95% C.L.) 
for the process e+e~ — > t+t~. Also shown are bounds anticipated from LEP2 
at levels of assumed precision as indicated by labels. Four different chiralities 
are considered. 
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Figure 9. Allowed regions of sin^ and M2 anticipated for the process e^e~ fl 
at the NLC at levels of assumed precision as indicated by labels. Four different 
chiralities are considered. 
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Figure 10. Cross section for bbveJ-'e production at i/i = 190 GeV, subject to 
cuts given by Eq. ()4.10j) . 
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Figure 11. Bounds anticipated from LEP2 for e^e~ hhvei^e at levels of 
assumed precision as indicated by labels. Also shown is the allowed region of 
sin^ and M2 obtained from LEPl data (95% C.L.) for the process e+e" — > hh. 
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Figure 12. Bounds anticipated from ^/s = 500 GeV for e+e" bbueiye at levels 
of assumed precision as indicated by labels. Also shown is the allowed region of 
sin^ and M2 obtained from LEPl data (95% C.L.) for the process e+e~ — > bb. 
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